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ABSTRACT 
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sense of Web-based data; monitoring of municipalities and providers of early 
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Monitoring: Proceedings of the 2002 National Monitoring Conference 

Introduction 



Purpose and Background 

Since 1992, Project FORUM at the National Association of State Directors of Special Education 
(NASDSE) has produced several documents on the topic of monitoring (Aheam, 1992, 1995, 
1998, 2000; Tschantz, 2002). Each of these documents includes an analysis of data from a 
national survey of state monitoring practices 1 and/or an in-depth report of one of the five 
National Monitoring Conferences. The most recent Project FORUM document summarized 
findings from the 2001 national survey and reported on the changes states have made in their 
monitoring practices from 1997 to 2001. This document is a report of the sixth National 
Monitoring Conference held in Salt Lake City, Utah in October 2002. It was compiled by Project 
FORUM through its Cooperative Agreement with the U.S. Department of Education Office of 
Special Education Programs (OSEP). 

Federal Monitoring of States 

Continuous Improvement Monitoring Process (CIMP) 

Since 1975, OSEP has monitored state education agencies (SEAs) for their compliance with Part 
B of IDEA. According to the 23 rd Annual Report to Congress on the Implementation of IDEA 
(U.S. Department of Education, 2001), OSEP’s accountability work has focused on “. . . 
improved results for infants, toddlers, children and youth with disabilities without sacrificing any 
effectiveness in ensuring that the individual rights of children with disabilities and their families 
are protected” (p. IV-1). Originally termed program administrative review, federal monitoring 
procedures have evolved over the years. The result has been the development and evolution of 
OSEP’s Continuous Improvement Monitoring Process (CIMP). CIMP is built on eight critical 
themes: (a) continuity; (b) partnership with stakeholders; (c) state accountability; (d) self 
assessment; (e) data-driven; (f) public process; (g) technical assistance; and (h) evidence of 
change that improves results for children with disabilities and their families. 

CIMP is an on-going process and includes: 

• self assessment; 

• data collection by OSEP in selected states; 

• improvement planning; 

• implementation of improvement strategies; 



1 Three of these national surveys have been conducted and are referred to as Profiles of State Monitoring Systems 
1994, 1997, 2001, respectively. The 1994 and 1997 surveys were conducted by the Regional Resource Center 
Network and coincided with the third and fourth National Monitoring Conferences. The 2001 survey was conducted 
by Regional Resource and Federal Centers Network and the National Early Childhood Technical Assistance System 
and was intended to coincide with the National Monitoring Conference scheduled for fall 2001. Due to the events of 
September 11, 2001 this conference was postponed, thus the conference proceedings are reported in this separate 
document. 
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• verification and consequences; and 

• review and revision of self assessment. 

OSEP customizes its monitoring based on the needs of individual states and organizes 
monitoring around the following four “clusters” for Part B (services for children ages 3 through 
21): (a) parent involvement, (b) free appropriate public education in the least restrictive 
environment, (c) secondary transition and (d) general supervision; and around the following five 
“clusters” for Part C (services for children ages birth through 2): (a) general supervision, (b) 
child find and public awareness, (c) early intervention services in natural environments, (d) 
family-centered systems of services and (e) early childhood transition. 

These cluster areas are used to organize the requirements of IDEA that provide the strongest 
links to improved results for infants, toddlers and students with disabilities. The specific 
activities conducted during each phase of monitoring are described in the 23 rd Annual Report 
(U.S. Department of Education, 2001, pp. IV-1-6). 

Focused Monitoring 

In addition to CIMP, the use of focused monitoring has been discussed at the federal level in 
recent years as “ ... a data-driven approach to monitoring characterized by focusing on a small 
number of carefully chosen priorities that have the greatest impact on improving results for 
infants, toddlers, children and youth with disabilities” (U.S. Department of Education, October 
2001). In 2000, a group of stakeholders met for the first time to begin the development of a 
focused monitoring model for OSEP to use when monitoring states. Several key elements of 
focused monitoring have been identified: (a) selection of a limited number of priorities, each 
supported by measurable indicators; (b) data/evidence-based decision making; (c) technical 
assistance to states in need of improvement; (d) the use of standard, uniform performance 
benchmarks when making monitoring decisions; (e) attention to diverse populations; (f) 
assurance that the state has an effective dispute resolution system; and (g) clear triggers for the 
initiation of interventions and sanctions. This group of stakeholders developed draft priorities, 
indicators, benchmarks, thresholds and triggers for focused monitoring. At the time of the sixth 
National Monitoring Conference, OSEP was looking at how to integrate focused monitoring and 
CIMP to create a strong accountability system that improves results for infants, toddlers, children 
and youth with disabilities. 

State Monitoring of LEAs 

Since its original passage in 1975, the IDEA has held SEAs responsible for monitoring how 
LEAs provide programs and services for students with disabilities. The following portion of 
IDEA, as amended in 1997, is the current legislative wording for the state supervisory role: 

(11) STATE EDUCATIONAL AGENCY RESPONSIBLE FOR GENERAL 
SUPERVISION- 

(A) IN GENERAL- The State educational agency is responsible for ensuring 
that- 

(i) the requirements of this part are met; and 
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(ii) all educational programs for children with disabilities in the 
State, including all such programs administered by any other 
State or local agency - 

(I) are under the general supervision of individual in the State who 

are responsible for educational programs for children with 
disabilities; and 

(II) meet the educational standards of the State educational agency. 

[20 U.S.C. Chapter 33, Sec. 1412(a)] 

Overview of the Conference 

The 2002 National Monitoring Conference was entitled, Showcasing Quality, Ensuring 
Compliance, Emphasizing Outcomes: A National Conference Addressing State Monitoring of 
IDEA. The overall purpose of the conference was to “share information about effective practices 
and to address state agency challenges in monitoring local programs providing services under 
Parts B and C of the IDEA.” The primary focus was state monitoring of local programs. OSEP 
personnel also participated and provided updates on federal issues, including upcoming changes 
in federal monitoring. The following topics were highlighted throughout the conference: 
effective practices in monitoring local programs; state accountability for results; data 
management approaches and uses; and state collaborative, focused and integrated monitoring. 

The conference was planned by the Monitoring Workgroup that included representatives from 
the Federal Resource Center (FRC), the Regional Resource Centers (RRCs), the National 
Association of State Directors of Special education (NASDSE), the National Early Childhood 
Technical Assistance Center (NECTAC) and OSEP. Proposals for conference presentations 
were solicited from all states, including SEAs and Part C lead agencies. All states were also 
invited to participate in the Marketplace of IDEAs — a sharing activity designed for states to 
display and describe their individual monitoring activities. Representatives from 49 states and 
two non-state jurisdictions attended the conference. The session titles and involved states are 
noted before each session summary below. 

Session Summaries 



Plenary Sessions 

Improved Outcomes Through Collaborative Monitoring 

Presenters: Barbara Schliesser, Director of Special Programs, NE 
Susan Wood. Coordinator, School Support System, RI 
Charlotte Lewis. Program Coordinator, NE 
Andrew Rikli. Program Specialist, NE 



Collaboration has been an integral part of both Rhode Island and Nebraska’s monitoring systems. 
In Nebraska, the Department of Health and Human Services Systems, in conjunction with the 
Department of Education, is implementing a Part C Quality Improvement Process that monitors 
special education programs and services. In Rhode Island, the School Support System is a 
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collaborative process among local school districts and state agencies designed to improve 
outcomes for students with exceptionalities. This joint session included examples of 
collaborative activities, lessons learned and changes in local programs as a result of monitoring 
findings. 

Nebraska: Nebraska’s Quality Improvement Process of monitoring local programs is built on the 
principle that the process is a collaborative activity and focuses on compliance with program 
requirements. The Nebraska Early Development Network is composed of two state agencies — 
the Department of Education (NDE) and Health and Human Services Systems (NHHSS) — and 
29 regions that collaborate to provide services to children with disabilities birth to 3 and their 
families. 

Each region has a local Interagency Coordinating Council, called a Planning Region Team that is 
made up of a lead agency, services coordination contractor, school district or educational 
services unit representatives, Head Start, child care providers, social workers, community 
representatives, clergy and families. School districts or Educational Service Units (ESUs) 
provide evaluation and education and related services (physical therapy, occupational therapy, 
speech/language therapy) to children. Medicaid, respite and services coordination support are 
provided by NHHSS. Nebraska chose a collaborative system of state supervision. A co-led team 
composed of managers, technical assistants, a family partner and monitors from NDE and 
NHHSS guide policy and implementation for the more than 500 school districts, 93 counties, and 
six health and human services areas. 

The monitoring system, as well as the Early Development Network, was designed with four 
principles in mind: (a) local decision making where possible; (b) family choices; (c) compliance; 
and (d) quality delivery system. The indicators and process were products of discussions by the 
NDE and NHHSS, OSEP, families and the Early Childhood Interagency Coordinating Council. 
The indicators and process are used to monitor early intervention, Medicaid waiver programs and 
family satisfaction. 

The monitoring process allows for as much local decision making as possible. Co-leads send a 
letter to the Planning Region Team introducing the process and consultants contact a local 
representative to establish the date for a site visit and a list of people who should participate in 
the orientation session. A Tool Kit and Sample Final Report are given to the team before the 
orientation session. This provides participants with a list of interview and survey questions and 
compliance items that will be reviewed. Local participants at the orientation session decide 
whom to interview and how information from families should be gathered (i.e., survey, home 
visit, interviews in person or phone or a mix of methods). 

Monitoring tools used during the monitoring process are file reviews and surveys. On the file 
review, indicators are marked in compliance , out of compliance or does not apply. Indicators can 
also be listed as a strength. Surveys are conducted with the school district, service coordination 
contract agency, service coordinators, providers, families, etc. and can also be conducted through 
focus groups or interviews. When the state monitors, surveys, focus groups and interviews are 
conducted to confirm findings from the file review. Typically, when a file review indicates non- 
compliance, the surveys are not positive and vice versa. If there is a disconnect between file 
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reviews and interviews, state monitors go back to find out why. Lessons learned from 
Nebraska’s monitoring process include: it is very time consuming and expensive; overall, it is a 
good process; and the tools are strong, but more thought is needed on different ways to do 
business, to bring Parts B and C together. 

Next steps — the NDE-ESU collaboration project. The purpose of this project is to establish a 
service delivery system and network to better serve families and children. Many services 
provided by NDE and ESU overlap. Goals of the project are to reduce duplication, better focus 
the ESU’s efforts and support to schools, and provide opportunity for those closest to the 
program to provide direct support. Project activities include assistance in continuous 
improvement review, professional development, conflict resolution, school improvement 
activities, complaint investigation, parent resources, technology support and support in data 
collection and analysis. This model of collaborative monitoring occurs through state level 
interagency cooperation (HHSS and NDE) and through state and local level cooperation (NDE 
and ESU). Nebraska is still in the process of determining how ESUs will interact with the local 
planning teams. 

Rhode Island : The Rhode Island Department of Education, Office of Special Needs 
(RIDE/OSN), is implementing the School Support System (SSS) to improve outcomes for 
students with disabilities who receive special education services. The SSS evaluates the quality 
and effectiveness of special education and the extent to which the district (local education 
agency) meets special education laws and regulations. It emphasizes: 

• involving the whole school district, including administrators, special and general educators, 
students and parents; 

• gathering and analyzing information from multiple sources of information; 

• visiting all schools, interviewing administrators and staff to focus on the appropriateness of 
the education for specific special needs students; and 

• developing a School Support System Plan for strengthening programs and 
correcting essential areas in order to improve student performance. 

The SSS is an evolving process, currently beginning its second full 5-year cycle. The Office of 
Special Needs, in conjunction with the Rhode Island Technical Assistance Project, has 
continuously refined the SSS in order to ensure alignment with the IDEA. The SSS also 
addresses the Comprehensive Education Strategy and the Rhode Island Student Investment 
Initiative. These are state general education initiatives designed to close gaps in student 
performance and prepare students for the 21 st century. The SSS is designed to align with current 
standards-based reform efforts and supports the following beliefs and assumptions: 

• An assigned category or level of disability does not define the educational needs of students. 

• To the maximum extent possible, students with special needs are meaningfully included in 
the general education program. 

• The curricula are based on standards that are sufficiently broad to support the learning needs 
of all students and include academic and skill areas. 
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• Individual Education Plans reflect state and local standards for student performance, 
incorporate varied assessments, and utilize a broad array of accommodations for testing and 
learning. 

• A comprehensive system of professional training must support and encourage the 
involvement of all personnel in addressing the learning needs of students with the full range 
of abilities and disabilities. 

The SSS integrates multiple sources of information in order to develop a support plan that is 
directed at increasing student performance and is founded on proven practice. The RIDE/OSN 
seeks to create collegial and collaborative relationships with school districts, involving the entire 
district in evaluating the quality of its special education services. As a result, the process 
delineates the district’s strengths and needs, culminating in the development of a plan to improve 
service delivery. The goal is to implement agreements in a timely and systematic way to get 
corrective actions instituted in order to assure continuous high performance of all children. 

What If Nothing Works? Are There Practical Sanctions to Ensure IDEA Implementation? 2 
Presenters: W. Alan Coulter. Director, National Center for Special Education 
Accountability Monitoring, LSU Health Sciences Center 
James Comstock-Galagan. Executive Director, Southern Disability Law Center 
Virginia Beridon, State Director of Special Education, LA 
Alice Parker. Assistant Superintendent, Special Education Division, Department 
of Education, CA 

Therese Y anan. Attorney, Native American Protection & Advocacy Project 

Alan Coulter opened the session by presenting two premises that guide the discussion and use of 
sanctions to ensure IDEA implementation. Premise 1: Most LEAs and service agencies are 
eager to effectively implement IDEA. Instances of non-compliance can be significantly reduced 
when requirements are explicit and well-understood and technical assistance/follow-up is 
provided. However, in some instances, sanctions are necessary to ensure implementation of 
IDEA. Premise 2: Sanctions are needed so everyone understands the seriousness of providing 
required services. Sanctions are about changing behavior so children receive needed services not 
so that punishment is applied to particular adults. There is little scientific data on the 
effectiveness of sanctions related to IDEA. 

Therese Yanan discussed the philosophy of sanctions, rewards and sanctions, the protocol for 
interventions or sanctions and types of interventions and sanctions. Overall, there is a trend 
toward sanctions — this is a fundamental principle of a focused monitoring system. The 
President’s Commission on Excellence in Special Education report states that sanctions should 
be a part of any accountability system. Under No Child Left Behind (NCLB), the state plan must 
include a single, statewide accountability system that ensures that schools are making annual 
yearly progress, including the use of sanctions and rewards. 



2 The visual presentation for this session is available online at the special education accountability website: 

www.hdc.lsuhsc.edu 
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Rewards and sanctions: The national stakeholders focused monitoring workgroup has discussed 
both the use of rewards if states exceed benchmarks and the use of sanctions if benchmarks are 
not met or if corrective actions are not taken within the established timeline. Examples of 
rewards include: a letter to the Governor; a national press release or other publicity on success; a 
plaque; commendation on OSEP’s website; banners like those used for Blue Ribbon schools; and 
documentation and funds for the state to record processes that were used successfully. 

Proposed protocol for interventions or sanctions: 

• Step one — analyze the state’s data to determine if there is a legitimate explanation for why 
the state has not performed as expected. If there is, OSEP will not intervene. 

• Step two — if failure is based on the state’s refusal to comply, then OSEP will impose a 
sanction. If failure is not based on a refusal to comply then proceed to next step. 

• Step three — OSEP will determine the cause for the failure, which will influence the level of 
sanction. Technical assistance (TA) or other interventions include assistance from the RRCs 
or other technical assistance centers, OSEP approval of state improvement plan, detailed 
technical assistance, assistance from high performing states and provision of data 
management resources. If there continues to be a failure to improve after TA, then sanctions 
would be imposed. 

Level and type of sanctions: The least coercive sanctions discussed by the stakeholders group 
include targeted resources to focus on capacity building and voluntary performance agreements 
with timelines outlined. Moderately coercive sanctions include designating the state as high risk 
and applying special conditions, compliance agreements, issuance of cease and desist order, 
issuance of conditional approval for state plan, referral to the office of inspector general, and 
referral to the U.S. Department of Justice. The most coercive sanctions include directing state 
officials regarding the areas in which funds should be expended, disapproval of a state plan, and 
partial or complete withholding of federal funds. 

Louisiana : Virginia Beridon discussed the process of using sanctions in her state. A continuum 
of interventions and sanctions has been developed for local districts in non-compliance. Districts 
that fall into sanction status have been in "focus" for three years in a row (based on Louisiana’s 
three statewide indicators) or whose follow-up visit revealed that all corrective actions had not 
been completed. 

• Level 1 — the SEA and LEA collaboratively design a Corrective Action Plan (CAP). 

• Level 2 — if the CAP did not work, the SEA works with the district to redesign its CAP to 
more effectively address concerns. 

• Level 3 — an intensive plan for technical assistance to address non-compliance is developed. 
This plan must be approved by the local school board and must be published. 

• Level 4 — the LEA is required to target IDEA funds to address the identified non-compliant 
findings. 

• Level 5 — the appointment of a special monitor or management team is required to oversee 
the corrective action improvement plan. 

• Level 6 — IDEA Part B funds are released on a conditional basis. The conditions are written 
into the corrective action improvement plan. 

• Level 7 — IDEA Part B funds are withheld with approval from the state board of education. 
There are approximately 80 LEAs in Louisiana. 
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Currently four school districts are at Level 2 and four are at Level 3. Sanctions are making a 
difference in Louisiana — there has been an increase in the amount of effort going into improving 
programs. 

California : Alice Parker discussed “Friendly and Not So Friendly Persuasion: California’s 
Enforcement and Sanctions Options.” California’s system of overall supervision and monitoring 
includes a balance between self-review and focused monitoring. Technical assistance (TA), 
corrective actions and sanctions are all used. Enforcement and sanction options include: public 
publishing and dissemination of LEA data summaries; monitoring reports; corrective action 
plans and monitoring reports; non-approval of local plans; requiring local boards of education to 
hold public hearings on noncompliance issues; sending letters to boards of trustees with a copy 
to the district advisory committees; issuing press releases; ordering compensatory services and 
reimbursement; requesting a writ of mandamus in a state court; granting awards with special 
conditions (they have to report quarterly); withholding federal Part B dollars (after a hearing 
with the Superintendent of Public Instruction); and stopping flow of all federal and state dollars. 
Out of total a 1200 complaints, 96 have required sanctions. NCLB provides for a stronger 
accountability system for all children. The concern is how to align a systems change with an 
individual entitlement statute. 

Jim Comstock-Galagan discussed the advocates’ views about effective sanctions. There are 
numerous rules children in schools are expected to follow and with real consequences when they 
are not followed. However, there are so few consequences for the individuals that run our 
schools. This is about power, but it should be about equity. The promise of IDEA remains just 
that. The National Council on Disabilities (NCD) found that not a single state is in compliance. 
Ask any room full of parents if this is true. The vast majority of people in schools want to do the 
right thing, but we must do better for children with disabilities. Accountability is needed for 
school districts and state departments of education. Enforcement has been difficult for SEAs. 
Educators are not trained to police compliance with the law — they are not comfortable in that 
role. They are, however, the state police in the education arena and they must do better enforcing 
the law. Remember, local schools have no difficulty enforcing their rules and imposing sanctions 
on students. SEAs needs to follow suit — it is one means of ensuring equity for students. 

When problems emerge, there is a need for very specific compliance requirements, corrective 
action goals and enforcement activities (sanctions). Enforcement activities should include three 
levels. 

• Level 1 — the SEA sends a letter of non-compliance to the LEA, which it must disseminate to 
all families with children with disabilities. The LEA noncompliance shall be the subject of a 
public hearing convened by the LEA board of trustees and shall be made a primary 
consideration in the evaluation of the LEA superintendent and special education coordinator. 

• Level 2 — when the LEA fails to correct noncompliance within six months, the following 
sanctions will be imposed: lowering accreditation of that LEA and suspending the 
certification of responsible administrative officials in the LEA (e.g., superintendent, special 
education coordinator). 



Monitoring: Proceedings of the 2002 National Monitoring Conference 
Project FORUM at NASDSE 



Page 8 
February 2003 



• Level 3 — the SEA withholds funds from the LEA or transfers money. These sanctions need 
to be automatic and mandatory. In order to change behavior, LEAs and top administrative 
officials must be held accountable for violations of not just rules, but the law. 

Focused monitoring holds great promise for students with disabilities because it involves an in- 
depth analysis of LEAs’ performance on key IDEA mandates. Focused monitoring produces 
sufficient information to support the types of enforcement activities outlined above. 

Monitoring Resources and the Monitoring Best Practices Web Site 

Presenter: Wayne Ball, Program Specialist, Mountain Plains RRC 

Resources available to states and lead agencies for monitoring and best practices were explained. 
If children are given the education that they need, monitoring will follow. A lot of what states are 
doing in monitoring is improving instruction and learning for children. The National Monitoring 
and Promising Practices website, where exemplary programs are identified, was put together to 
assist states in these efforts. The website is housed at the FRC (a tour of the website was 
provided during this session): http://www.dssc.org/frc/nmpp/index.litm . 

The website includes “Monitoring Profiles.” Each state has submitted a profile for Part B 
monitoring. For Part C monitoring, only about 13 states have submitted one. It would be helpful 
if more states submitted for Part C. OSEP resources are available through the website, including 
the CIMP manual. Also there is a link to the OSEP monitoring website which includes self 
assessments, improvement plans for states, state contacts (find out who your OSEP contact is) 
and CIMP clusters and indicators. Many states have shared promising practices and monitoring 
issues via the website. More needs to be added, especially more on promising practices. There 
also are links to OSEP’s technical assistance (TA) network. The Regional Resource Centers 
(RRCs) are a great resource for states, including training materials. Each RRC has an 
information specialist. 

Make this information available to your local school systems. A lot is known about promising 
practices but this information needs to get out to the local schools. If proper instruction and 
learning are provided, monitoring will take care of itself. 

OSEP Plenary Sessions 

Lunch Address 

Presenter: Stephanie Lee. Director, OSEP 

The title of this conference. Showcasing Quality, Ensuring Compliance, and Emphasizing 
Outcomes, covers three very important themes as we work together to improve outcomes for 
children with disabilities. With the passage of NCLB, we are focusing on accountability for 
results, reflecting the President’s view that every child counts. Schools should be held 
accountable for all children. Accountability for results is one of four components of NCLB and 
will also impact the reauthorization of IDEA. Accountability for results was also part of the 
President’s Commission on Excellence in Special Education report. The Commission’s report 
discussed federal monitoring on the implementation of IDEA and recommended changing the 
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monitoring process to focus on performance and results. There is a need for continuous 
improvement related to compliance findings, encouraged flexibility and innovation, and a shift in 
the focus of monitoring from one on process to one on results. Through CIMP, OSEP has been 
moving to a focus on results since 1997. Monitoring must be a process to ensure accountability 
and services that result in outcomes. In your role, this shift will create challenges. 

OSEP monitoring themes that could be incorporated into state monitoring include: 

• Working in a collaborative manner versus an adversarial one. Critical to this collaboration is 
supporting state efforts to have steering committees that include a diverse group of 
stakeholders as well as encouraging internal collaboration (e.g., Part B and C within states 
and between general and special education). 

• Data-based decision making. A variety of data sources must be used and must be applied in a 
structured fashion. 

• Importance of technical assistance resources and supporting improvement efforts. Available 
resources include OSEP’s technical assistance and dissemination projects, resources on 
specific topics (e.g., accessing the general education curriculum, transition and due process), 
personnel preparation grants and general supervision enhancement grants. The TA network is 
designed to get research to states and then we count on the states to get the information to 
local school districts. 

• Accountability and enforcement. This includes declaring a state as a high-risk grantee, 
identifying persistent non-compliance and developing compliance agreements. OSEP staff 
will provide more information on accountability and enforcement in later sessions at this 
conference. 

OSEP is pleased with its partnership with the National Focused Monitoring Workgroup. 
Although its written products do not represent the position of the Department of Education, the 
Workgroup has had a positive impact. Focused monitoring does not replace CIMP; rather it 
strengthens our monitoring by focusing on those indicators that have the strongest link to 
improving outcomes for children and families. 

CIMP Monitoring: Lessons Learned? 

Presenters: Ruth Ryder. Division Director, Monitoring and State Improvement Planning 
(MSP), OSEP 

Rorie Lackey, Assistant Director for Special Education, NV 
Gene Lenz. Senior Director of Special Education, TX 
Anne Lucas, Coordinator, Infant and Toddler Connection of VA 
Marilyn Pearson, Assistant Director for Special Education, MT 

Ruth Ryder introduced the panel of state staff. Each of these states shared its experiences with 
CIMP by responding to the following questions: 

1 . What were your expectations going into the CIMP self assessment or improvement planning 
session? 

2. What were the strengths of the process? 



3 The visual presentation from this session is available online at the National Monitoring and Best Practices Website 
(look for the National Monitoring Conference link) at: http://www.dssc.org/frc/MonitorConf.htm 
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3. What were the challenges of the process? What resources were needed to address these 
challenges? 

4. What didn't work as well as expected? What would you do differently? 

5. What are the next steps for your state’s program? 

Virginia’s Part C Experience - Expectations of CIMP: CIMP was an additional time-consuming 
task, but also a positive means for improving systems of services. Strengths of the Process : The 
focus on self assessment and data-driven decision making enhanced the state’s existing process 
of systems improvement. CIMP built on the local monitoring process and developed linkages 
and interrelationships. The logic model analysis assisted in focusing on results, not just quick 
fixes. TA from Mid-South RRC and NECTAC was helpful. 

Challenges of the Process: Timing was a challenge. Virginia did not have an opportunity to 
integrate CIMP with the existing state improvement process. Previous improvement efforts were 
put on hold and there were perceptions that the previous work was not important. The logic 
model training was conducted after the self assessment. What To Do Differently: Introduce the 
logic model up front. Provide sufficient lead-time to states to ensure a carefully planned 
approach to initiating self assessment, including time to integrate existing systems improvement 
efforts. 

Montana’s Part B Experience - Strengths of the Process: The involvement of stakeholders in 
decision making was a strength. The state continues to use the advisory panel as the steering 
committee. It reviews improvement plan activities and progress. The state structured the process 
and, as a result, had ownership of the design and strategies used. Other strengths included 
designing improvement strategies tailored to the state, data-informed decision making, and 
partnerships with Part C and other agencies. 

Challenges of the Process included access and availability of data. There were data elements 
required for the self assessment that were not collected. Another challenge was the validity and 
reliability of data. Procedures for verification of data were not in place for all data, like it is for 
child count. Other challenges included time constraints and funding. There was not enough time 
to gather and analyze all the data wanted. There were not enough resources to address all areas in 
improvement plan. Resources Used to Address Challenges include RRC, OSEP, state staff and 
Montana’s CSPD/SIG. (Fortunately, the SIG was approved following development of the State 
Improvement Plan. The SIG has been instrumental in providing the funding to implement 
improvement strategies identified in the State Improvement Plan.) 

Next Steps include improving the quality of data; identifying additional data elements needed to 
assist in self assessment and review of progress on performance indicators; conducting a self 
assessment (not waiting for notification from OSEP, this is a continuous process); refreshing the 
Improvement Plan; and linking key indicators from the state to the local level. 

Nevada’s Part B Experience - What Worked: Quality facilitation — having someone 
knowledgeable about the special education system at the state and national levels, with the 
ability to help individuals understand how to use data to make decisions and the skills to help 
groups reach consensus. Stakeholder involvement — leadership roles were created for the PTI 
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director and State Advisory Committee chairman as steering committee co-chairs. Other 
strengths included using cluster area teams and steering committee membership from essential 
groups; public input through surveys and focus groups; and a focus on partnerships to reach 
results. 

Challenges: Show me the data — districts did not necessarily have data the state needed. Timing 
is everything — often felt as if they were chasing moving targets and waiting for feedback from 
OSEP. Solutions are not always in the state’s hands and do not come easy given high standards 
(underscored by NCLB) and mixed priorities. Hindsight is 20/20 : People get burned out — the 
same people may be asked too often to serve the cause. Traveling to meetings takes time and 
causes anxiety for some people. Technology needs to be used effectively while preserving 
quality interpersonal dynamics. Using data may require a shift in thinking. What's Next? 
Planning is only the first step — “we've got our marching orders, now let's march”. The state’s 
ongoing process will continue to be evaluated. Improvement strategies have been developed and 
now the state needs to implement them. 

Texas’ Part B Experience - Expectations: Texas was ready for change and CEVEP served as the 
catalyst. Improvement was the goal. Federal monitoring with a purpose made it positive. 
Strengths of the Process: The process can be customized to meet an individual state’s needs. 
Challenges: There were challenges with some of the indicators - many of them were confusing. 
Questions at public meetings were very broad. The process is time- and staff- intensive. 
Outsourcing public meeting facilitation and steering committee facilitation was positive for the 
process however, effort had to be made to keep staff involved with and owning the process. 

What would you do differently ? States must spend quality time with the steering committee to 
teach them the system. Begin immediately; you can't start early enough. All of us collectively are 
swimming in data, but we are not swimming in information with which to make decisions. 
Remain focused. Next Steps: Texas has already started the second round of public meetings and 
is waiting for OSEP’s report. The Texas Continuous Improvement Process includes an 
Improvement Planning Committee, the Texas Steering Committee and a system of continuous 
review. The steering committee members have been asked to serve on separate improvement 
committees, focused on specific areas. This has broadened involvement and the number of 
experts on those topics. The process has been integrated into everything the state is doing now. 
Words of Wisdom: Perspective is what allows us to step back and see the entire forest . . .the big 
picture is important! 

Focused Monitoring: A Novel Approach 

Presenters: Ruth Ryder. Division Director, MSDP, OSEP 
Larry Wexler . Deputy Director, MSDP, OSEP 

Therese Y an an . Attorney, Native American Protection & Advocacy Project 
John Copenhaver . Co-Director, Mountain Plains RRC 
Tal Black, Associate Director, NECTAC 

Ruth Ryder briefly described the National Monitoring Stakeholder Group and representatives 
from that group discussed focused monitoring. 
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Larry Wexler outlined the historical perspective on federal monitoring. In 1998, OSEP supported 
a meeting of stakeholders, including technical assistance providers, Department of Education 
staff, SEA staff, parents and advocates to discuss how to make monitoring focus on improving 
outcomes. The result was a continuous improvement approach with six principles for 
monitoring: partnership with stakeholders; self assessment; data used to drive decisions; 
extensive public input; OSEP providing TA as needed; and improvement planning focused on 
outcomes rather than process. 

The Department took this information and developed and implemented CIMP. Based on input 
from SEAs, advocacy groups and the Commission’s report, and given OSEP’s limited resources, 
there is now a move toward focused monitoring. The National Monitoring Stakeholders Group 
convened a series of meetings. From these meetings the following emerged: agreement on a 
concept of focused monitoring; monitoring based on data-driven decisions; and a better 
understanding of what data are available and in what format. The group developed draft 
priorities, indicators, benchmarks (used to determine non-compliance), thresholds and triggers 
(used to determine when to initiate action). OSEP is now looking at how focused monitoring and 
CIMP come together to create the best system for kids. 

Therese Yanan shared her experiences as a member of the National Monitoring Stakeholder 
Group and the New Mexico Stakeholder Group. The great thing about the stakeholder group is 
the level of collaboration. It increases the level of understanding about challenges faced by 
individuals. Collaboration also increased understanding about the amount of data collected and 
the amount not being used — which is one benefit of focused monitoring. The reality that there 
was more agreement than disagreement within the stakeholder group was a surprise. 

It helps to understand each other’s perspective and to be clear on the role of the stakeholder 
group. The focus of each meeting needs to be clear and it is critical to establish a time when the 
group will meet again. E-mail and websites can be helpful to communicate this information. 
This process is time consuming. It takes time to build relationships and see the impact of the 
process. 

Parents and advocates know it's going to take time before anything happens. If people understand 
where they are and where they are going, it is helpful. Educators need to understand that parents 
feel like they don’t have a lot of time and results are important. 

Advocates and parents feel like they are giving up something being part of a focused monitoring 
stakeholder group (e.g., selecting three indicators when you have a number of issues/concems). 
The education system has historically been out of compliance and children are not being served. 
All parents have to hold onto is the process that IDEA requires. If the monitoring system changes 
without improving education for students with disabilities, then parents and advocates are going 
to have a more difficult time trusting schools and educators. 

John Copenhaver discussed some of the promises and challenges of the focused monitoring Part 
B priorities and indicators. There are six priority areas for Part B focused monitoring: (a) 
effective state supervision; (b) meaningful and effective family involvement; (c) development 
and performance of outcomes for children and youth with disabilities; (d) inclusion of children 
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and youth with disabilities in typical community and school setting with their non-disabled peers 
with needed supports; (e) effective transitions; and (f) enhanced social, emotional and academic 
development for children and youth with disabilities through the use of positive behavior 
supports and strategies and improved school climate. 

Positive Aspects of Focused Monitoring include the limited number of indicators, the focus on 
critical aspects of the regulations and the use of data and information. Focus priorities become 
the content and CIMP is the process. Priority-indicator-benchmark-trigger makes sense. The 
concept of triggers has promise if data issues can be solved. Focused monitoring allows limited 
resources to focus on priority issues and aligns focus priority indicators with CIMP cluster areas. 

Challenges'. Data are not available on many indicators. Data on some indicators are collected 
from other sources but are not consistent from state to state. Some indicators are interpreted 
differently according to state policy and procedures (e.g., graduation rate). Maintaining 
stakeholder involvement is difficult. Terminology is open to different interpretations. Comparing 
states on certain indicators will be a challenge. 

Data Issues : In many cases we are comparing apples and oranges. For example, states have 
different methods for determining graduation and dropout rates (some states count students 
working toward a GED while others do not). If the data are not clear, there are problems. Data 
are currently collected for only 7 of the 26 Part B focused monitoring indicators. 

Greatest Challenge : If not careful, focused monitoring will become top-down, clinical and OSEP 
driven, similar to the old compliance model. The best from CIMP and focus monitoring can be 
combined, retaining stakeholder involvement and state accountability. Greatest Promise : 
Focused monitoring can enhance the attention to those issues that are aligned with positive 
results for children with disabilities. This is a huge promise. CIMP cluster areas and focus 
priority areas need to be bridged and measurable indicators that make sense need to be 
developed. 

Conclusion: This is an opportunity to combine the best from CIMP and focused monitoring. 
CIMP encourages stakeholder involvement and state/LEA ownership of the process and focus 
monitoring provides direction on the issues most important for showing results for children with 
disabilities. Care must be taken not to create a complicated and bureaucratic process like the old 
compliance model. 

Tal Black discussed five focus priorities for Part C in the proposed focused monitoring of IDEA: 
(a) effective state supervision of child find; (b) meaningful and effective family involvement; (c) 
development and performance of outcomes for infants and toddlers with disabilities and their 
families; (d) embedding early intervention services in families’ daily routines and typically 
occurring community activities; and (e) effective transitions for infants and toddlers with 
disabilities (transition to Part B). While key indicators have been proposed for each priority 
area, some as yet lack the data necessary to set meaningful benchmarks and triggers. 
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There are some unique challenges in applying focused monitoring to Part C, such as: 

1 . How can the discretion provided to, and exercised by, the states under the Part C legislation 
be reconciled with the need for consistent data across states? 

2. What are the fundamental outcomes for children that Part C should be achieving and what 
would be viable measures for whether or not those outcomes are being achieved? 

3. What are fundamental outcomes for family members, especially parents and care givers, that 
Part C should be achieving and what would be viable measures for whether or not those 
outcomes are being achieved? 

4. Given the relatively brief amount of time children and families may spend in the Part C 
program what is reasonable to expect of Part C’s early intervention? 

The application of the focused monitoring approach to Part C offers a two-way challenge. Part C 
is challenged by the need to provide meaningful and reliable data in order to determine if 
expectations are being met. Focused monitoring is challenged by the uniquenesses of the Part C 
program within IDEA. But Part C is facing a fundamental accountability challenge and there is 
important work to be done. It has been 16 years since Part C (then Part H) was signed into law. 
Accountability questions concerning Part C are going to be asked and answered by someone. If 
we, in the Part C program, find those questions and answers to be inappropriate or unfair, but 
have contributed nothing substantial to the accountability dialogue, then we would have no one 
to blame but ourselves. In that sense, focused monitoring may be just the accountability wake up 
call Part C needs. 

Breakout Sessions: 

Quality Assurance Information System: Electronic Database for Tracking Compliance, 
Complaints and More 

Presenters: Rebecca Cort . Coordinator Statewide Special Education Quality 
Assurance, NY 

Daniel Johnson . Upstate Coordinator, Special Education Quality 
Assurance, NY 

This session described New York’s statewide electronic database (QAIS) for recording, tracking 
and generating reports regarding monitoring reviews, complaints and other compliance issues. 
This regionally based system records and tracks information relative to: (a) all types of 
monitoring reviews; (b) 60-day complaint investigations; and (c) information on key 
performance indicators related to classification, LRE placement and disproportionate 
representation. These data are used to prioritize districts for monitoring, identify the level of 
review districts will receive, identify regional and statewide issues for targeting technical 
assistance (TA) and to track timely completion of monitoring, TA and other compliance 
activities. Examples were shared of how district level reports/profiles can be generated and some 
of the system’s current problems (e.g., the need for a faster and bigger platform) were discussed. 
Other strengths of the system identified were the ability for regional and state staff to generate 
regional profiles of compliance issues in order to identify, compare and contrast prevalent 
compliance issues statewide. Resources are then targeted based on these results. The presenters 
noted they are working toward a system that will allow submission of monitoring reports directly 
from the field once the system is upgraded. 
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Defining Roles and Responsibilities of State and School Level Steering Committees 
Presenter: John Copenhaver, Director MPRRC 

The IDEA was last reauthorized in 1997. Shortly after the reauthorization, OSEP encouraged 
state education agencies (SEAs) and Part C lead agencies to assemble a representative group of 
stakeholders from throughout the state to assist in developing an IDEA implementation plan that 
would outline how the SEA lead agency was intending to implement the new law and 
regulations. 

This approach of planning and empowerment was extremely successful. The next year, OSEP 
pioneered its new CEMP. States were encouraged to utilize a “Steering Committee” to assist the 
SEA lead agency in conducting a self assessment and to be involved with improvement planning. 
The steering committees were made up of stakeholders and consumers of special education from 
throughout the states. Many states utilized their State Special Education Advisory Panel and 
Interagency Coordinating Council as their steering committee membership. Again, OSEP 
received positive feedback to this approach. Federal monitoring was moving from an “expert 
model” to an “empowerment model.” 

As states began to revamp their own systems, most evolved into processes very similar to the 
OSEP model. Many states encouraged local school districts to assemble steering committees to 
conduct school district self assessments and improvement plans. As the focus monitoring 
becomes part of the continuous improvement process, steering committees at the state and local 
levels remain extremely important. 

There are many benefits for the SEA lead agency when utilizing school district steering 
committees as part of the improvement and monitoring process, including: providing an 
opportunity for input from consumers and stakeholders; giving the educators and parents (those 
in a better position than the SEA lead agency) the opportunity to identify deficiencies and 
strengths; offering another data flow to the State steering committee; allowing improvement and 
monitoring to be more meaningful, with a sense of ownership; providing more opportunities for 
the SEA lead agency to provide technical assistance; and seeking involvement from general 
education and other consumers. 

The challenges include: deciding the steering committee membership; establishing meeting 
times/locations that are beneficial for all members; determining a funding source for steering 
committee activities; providing orientation and understanding of the process; finding enough 
members for very small school districts; and finding individuals with expertise in data collection. 

Suggested members for school district steering committees include: parents of children with 
disabilities; students with disabilities; special education teacher(s); related service providers; 
paraeducators; Title I staff; school principal; special education director; general education 
teacher(s); private school representative; and Part C/preschool representative. 

The following are “lessons learned” from a SEA lead agency as it implemented a continuous 
improvement and/or focus monitoring approach with the utilization of local school district 
steering committees. 
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DO 


DON’T 


• Provide orientation to steering 

committee members on their roles and 
responsibilities. 


• Create a complicated and bureaucratic 
self-assessment process. 


• Establish a balance of membership 
between general and special education. 


• Expect all local steering committees to 
be the same. Value school membership 
differences. 


• Develop a data flow from the school 
steering committee to the state steering 
committee. 


• Expect a lot of data to be available at 
the local level. 


• Involve individuals who represent the 
culture of the community. 


• Treat urban and rural school districts 
the same. 



Factors that seem to create successful school district steering committees include: 

• empowering parents and other stakeholders with information about the process; 

• providing a user-friendly and comprehensive orientation; 

• explaining roles and responsibilities of each stakeholder group; 

• providing incentives (travel expenses, child care, lunch, refreshments, acknowledgment); 

• describing the difference between “advisory” and “advocacy;” 

• implementing a consensus-building process; 

• involving representation from cultural groups in the state/community; 

• engaging and keeping informed key SEA/LA or school district administration; 

• providing a three ring binder of materials for each member; 

• using a trained facilitator; and 

• including a data person. 

Regional Self Assessment to Continuous Quality Improvement: New Hampshire ’s Part C Quality 
System 

Presenters: Carolyn Stiles . Part C Program, NH 

Jenny Todd, Intake Coordinator, Moore Center Services, Inc., NH 

Carolyn Stiles presented an overview of the values of New Hampshire’s self assessment for local 
monitoring of Part C and the Part C system structure, including use of a collaborative partnership 
approach to monitoring. Regional teams make decisions on improvement needs and the central 
focus is on responsive family-centered services. Data are used for decision making and regional 
data and information align with the statewide CIMP. System structure : The Part C lead agency is 
New Hampshire’s Department of Health and Human Services, which contracts for Part C 
services with 1 2 area agencies. Area agencies contract with 23 local programs. 

The regional self-assessment process includes: 

• a regional team with all stakeholder groups represented; 

• three surveys (family, service provider, community provider); 

• data gathering; 

• identification of regional strengths and areas in need of improvement; and 
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• development of an action plan for improvement. 

Jenny Todd shared her region’s experience with this process. Based on recommendations from 
the state, her region pulled together a team that included service providers, an early intervention 
director, community partners, service coordinators and parents. They sent out 200 surveys to 
parents, 80 to community partners and 1 6 to service providers. They analyzed survey data, came 
up with some goals and developed an action plan. Going through the process the second year 
was so much easier and smoother than the first year. Having technical assistance available from 
the state was very important throughout the process. Also, during the second year, surveys were 
on disk, which was helpful in terms of analyzing data. A regional improvement plan was written 
based on results of the self-assessment process. It is helpful at the local level to be able to do 
these. No major surprises with the process were reported. Overall it was very positive. 

Carolyn Stiles closed the session by sharing a lesson learned — limit the number of questions on 
the parent survey. New Hampshire attempted to do this, but when the team solicited a group to 
reduce the number, no one could really remove any questions. There is a need to focus on 
transition to Part B. Problems emerged around data and data collection. They did not take into 
account the differences in software used throughout the state. New Hampshire has five different 
sources of information and the hope is to consolidate the data. This will enable a comparison of 
data and regions. 

Looking Back and Moving Forward - Utah ’s New Special Education Monitoring Process 
Presenters: Karen Kowalski . Monitoring Specialist, UT 

Margaret Lubke de Arce. Research Assistant Professor, Utah State University 

Utah has revised their procedure for monitoring special education programs to closely model 
OSEP’s Continuous Improvement Monitoring Process. The presenters walked participants 
through a review of Utah’s previous monitoring system and presented a preview of their vision 
for a new system. The new approach is focused on improving efficiency through digitalization 
and use of a decision-making logic model. The new system intends to look for results in quality 
of procedures as well as actual student outcomes. School districts conduct a self assessment of 
programs to identify strengths, compliance issues and areas for improvement. An SEA team 
validates findings and an improvement plan is developed for compliance and student outcome 
issues. Utah will implement a focused monitoring approach, using strategies with the greatest 
potential to improve results for children, while at the same time keeping important procedural 
elements. 

Applying a Logic Model to Local Data in Monitoring for Better Outcomes for Children and 
Families 

Presenters: Jeff Harlow , Director of Social Science Research Center, Old Dominion 
University, VA 

Anne Lucas, Coordinator, VA Part C 

Mary Anne White , MIMS Coordinator, VA Part C 

Virginia’s Infant and Toddler Connection developed a logic model approach to its Monitoring 
and Improvement Measurement System (MIMS), to build capacity in local systems to produce 
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“meaningful plans for improvement.” Challenged by decentralization, introducing logic model 
thinking has been a successful approach to actively engage the state’s 40 local Interagency 
Coordinating Councils (ICCs) in processes yielding results-driven plans for improvement 
(plans). 

MIMS is dynamic and changing in response to state and federal priorities, CIMP initiatives, 
changes in Part C (federal and state) and improvements suggested through monitoring 
implementation. Virginia provides consultants for each of its MIMS and TA components. These 
staff members have separate responsibilities, but also specific connections that ensure local 
council assistance is consistent, continuous and aligned with self assessment and local plans. 
Supporting these is the increasing integration of Virginia’s multiple automated data systems so 
that local entities can retrieve and use the data necessary for sound decision making. 

The logic model approach enables local entities to focus on results and simultaneously ensure 
system elements are in place. It serves as an analysis tool, which guides identifying, ranking and 
strategizing issues and actions for the plans. The MIMS cycle begins with self-assessment 
planning and conducting, state team review and visit, development of the local plan and follow- 
up. Recent changes based on experience and feedback include: increased TA during local self 
assessment; re-timed self-assessment submissions and on-site visits to increase data currency; 
added training and tools to improve communication; and 30 percent fewer MIMS indicators. 

Just three years old, several impacts of the MIMS have been cited by localities and state Part C 
personnel, including collaborative working relationships, system approaches and issues, 
increased consistency in information and greater responsiveness to local needs. “MIMS is not an 
event, it’s a culture.” 

Making Functional Sense of Web-Based Data: Another Step Toward Better Outcomes for 
Children and Families 

Presenters: Jeff Harlow , Director of Social Science Research Center, Old Dominion 

University, VA 

Anne Lucas. Coordinator, VA Part C 

Virginia’s Infant and Toddler Connection developed the Infant and Toddler Online Tracking 
System (ITOTS), a web-based data system that currently stores and reports individual child 
intake data and will soon store and report service coordination and service delivery data. ITOTS 
was developed for three main purposes: (1) to increase the accuracy and completeness of data 
submitted by localities; (2) to ensure that accurate data are available to state and local decision 
makers in a timely fashion; and (3) to enhance service coordination functions. The first phase of 
ITOTS includes all federally required data and intake data collected at or before Individualized 
Family Service Plan (IFSP) meeting. Standardized and customized reports can be generated at 
the state and local level. The second phase of ITOTS will add three general functions: (1) 
capturing and reporting data related to the entire service delivery system; (2) providing online 
management tools to support the service coordination function; and (3) assisting providers with 
compliance. The data elements will include service units expected; service units delivered; 
service unit cost; amount billed for service unit; amount reimbursed; all relevant dates; and 
procedural safeguard milestones. 
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ITOTS is a source for relevant, accurate and timely data at the individual family level that is 
useful for both MIMS (Virginia’s Part C monitoring model) and CIMP monitoring. It provides a 
point-in-time picture of how the locality is performing in regard to each family, and aggregated 
data at the local and state level. It will facilitate comparisons over time at the individual, local 
and state level. A data system is currently being developed to integrate multiple data sources 
(including ITOTS) to develop electronic profiles for each locality. This integrated system will 
serve as a tool for the state in ensuring local accountability, determining local access and 
management of resources, determining local need from additional funds, and in providing profile 
and aggregated data for MIMS and CIMP. Additionally, this integrated system will verify and 
complement local MIMS data, organize state level aggregate data for CIMP, enhance state level 
improvement planning, evaluate the effectiveness of state improvement planning strategies, and 
assist in managing finances and resources. 

The West Virginia Continuous Improvement Monitoring Process: Planting a System ’s Change 
Presenters: Kav Johnston. Coordinator, Office of Special Education, WV 
Jim Parker . Coordinator, Office of Special Education, WV 
Judy Robertson. Coordinator, Office of Special Education, WV 

The presenters provided an overview of West Virginia’s monitoring process over the past two 
years and a description of the state context and school demographics. Prior to 2000, monitoring 
was compliance oriented, with no accountability component for student results. There was little 
involvement of LEAs in decision making for corrective actions. LEAs were also trapped in the 
conundrum of too many IEP violations. 

Timeline of the change process : In 1999-2000 the SEA developed a self assessment based on 
OSEP’s model. In Spring 2000, extensive training was conducted. In Summer 2000, the SEA 
provided technical assistance on the self assessment process. In September 2000, self 
assessments from each LEA were due. For 2000-2003, validation of self assessments has been 
incorporated into on-site monitoring visits. Progress reports on plans are due each November. 

West Virginia’s current monitoring system includes the following characteristics: (a) increased 
accountability for student results; (b) all schools visited; (c) increased accountability at the 
school level; (d) LEAs involved in the process through self assessment and improvement 
planning; and (e) on-going evaluation and continuous improvement. There are five components 
of the monitoring process: self assessment; data analysis by the Office of Special Education; on- 
site review; improvement planning; and the monitoring report. The monitoring instrument has 10 
self-assessment standards and 40 indicators. 

Monitoring information in West Virginia has been used in developing the recent OSEP self 
assessment, professional development activities, the state strategic planning process, unified 
school plans, tying discretionary grants from the SEA to CIMP, and in developing the Biennial 
Performance Report. 
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Focused Monitoring Part 1: Context, Data Management, Sample Analyses and LEA Profiles 
Presenters: W. Alan Coulter. Director, National Center for Special Education 
Accountability Monitoring, LSU Health Sciences Center 
Mary Hudler , Assistant to the Director of Special Education, CA 
Eileen Amy , Administrator, Bureau of Instructional Support and Community 
Services, FL 

Virginia Beridon . Director of Special Education, LA 
Trudy Potts , Director of Focused Monitoring, NM 
Chris Drouin . Special Education Consultant, CA 
Lalit Roy , Special Education Section Chief, CA 

Karen Denbroeder. Program Supervisor, Bureau of Instructional Support and 
Community Services, FL 
Denise Stewart , Consultant, FL 

Batva Elbaum . Research Assistant Professor, University of Miami, FL 
Liz Leger . State Coordinator for Focus Monitoring, LA 
Donnalee Ammons , Monitoring Consultant, LA 

Alan Coulter began the session with a definition of focused monitoring and an overview of the 
three focused monitoring presentations. The purpose of these sessions was to provide 
information about the activities of four states (CA, FL, LA, and NM) implementing focused 
monitoring. The visual presentation from these sessions is available online at the special 
education accountability website: http :// www .hdc.lsuhsc.edu / (Click on National Monitoring 
Conference in Salt Lake City). This first session focused on data, data management and 
individual system profiles. 

Louisiana: Louisiana moved toward a focused monitoring approach for several reasons: 
previous monitoring had not brought about change and was not focused on improving student 
outcomes; the federal emphasis on performance outcomes in the general education curriculum; 
and a statewide accountability initiative. Due to a court order, Louisiana had a special education 
data system in place prior to starting focused monitoring. The state also has a student information 
system for general education and collects statewide achievement data. The data are entered into a 
data management system from the local level to the state level. Performance Profiles of each 
school system are disseminated annually. The profiles include 12 data elements related to student 
outcomes, chosen by the state stakeholders group. Three of the data elements are chosen as 
“focus indicators” for ranking the school systems each year. 

Florida: The context for changing monitoring in Florida included dissatisfaction with the “old” 
process, limited resources, a desire for a meaningful system that supports improved student 
results and the need to connect with the State Improvement Process. Florida’s focused 
monitoring triggers were developed based on guidance from a stakeholder workgroup. They 
include regular class participation, dropout rate for students with disabilities, graduation with a 
standard diploma and participation in assessments. Florida’s LEA profiles are used for 
continuous and focused monitoring. LEA profile features were described and data quality issues 
were discussed. 
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California : Based on a desire to develop a monitoring approach that was more outcome- focused, 
a stakeholder group developed key performance indicators (now 10). These indicators are used to 
inform all monitoring and technical assistance activities. Twice a year data are entered into the 
California Special Education Management Information System (CASEMIS) for all children with 
an IEP or IFSP. This is not just a data system, but also an infrastructure where other data 
elements can be added. The flow of data to and from CASEMIS was discussed as well as the 
selection of districts for monitoring. California is currently developing benchmarks and targets 
for the key performance indicators. 

New Mexico : The goal of New Mexico’s current approach to monitoring is to help the system 
better enable students with disabilities to benefit from, and make progress in, their educational 
programs. New Mexico has an accountability data system (ADS) to gather data from all school 
districts. A stakeholder group developed key performance indicators and discussed data that 
needs to be collected. Each year a stakeholder group chooses focused indicators and there is the 
potential for these indicators to change every year. For 2002-2003, the indicators are: least 
restrictive environment; evaluation/identification of specific learning disabilities; student 
performance; and test participation. Based on these indicators, LEAs are designated for 
monitoring purposes as focus LEAs, random LEAs, or continuous improvement LEAs. 

A Statewide Quality Improvement Monitoring System of Municipalities and Providers of Early 
Intervention 

Presenters: Denise Berletic . Health Program Administrator, Early Intervention, NY 
Mary Lou Clifford . Public Health Program Nurse, Early Intervention, NY 

The New York State Department of Health, Early Intervention Program conducts quality 
improvement monitoring reviews of municipalities administering local early intervention 
programs (58) and providers of early intervention services. The state contracts out monitoring 
responsibilities. Fourteen municipalities, 60 provider agencies and 120 individual providers are 
monitored each quarter. A data system is being developed to automate scheduling, collect data 
during the review process, produce the reports and provide data to the state. Review tools are 
programmed into this system and teams can enter their findings while on site. 

Pre-review procedures include the following: telephone surveys with parents; surveys of counties 
and providers; self assessment by provider/municipality prior to on-site visit; and review of 
parent survey data, previous monitoring reports and corrective action plans (CAP). On-site 
review procedures include entrance interview, observation of facility, review of child records and 
additional documentation, staff interviews and exit interviews. Review tools have several 
purposes: (a) to gather data from various sources; (b) to include interactions with providers, 
municipalities and parents to assess practices; and (c) to ensure compliance with the state’s early 
intervention program regulations. The post review activities include drafting a monitoring report 
and developing a CAP. The provider/municipality has 30 days to respond to the draft report. The 
state will provide assistance in developing the CAP if needed. Once the CAP is accepted and all 
comments received, the report will be finalized and made public. 
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The North Carolina Continuous Improvement Monitoring System (CIMS) 

Presenters: Mark Gupton . Monitoring Consultant, Exceptional Children’s 
Division, NC 

Ira Wolfe , Monitoring Consultant, Exceptional Children’s Division, NC 
Bobbie Grammer. Monitoring Consultant, Exceptional Children’s 
Division, NC 

This session provided an overview of North Carolina’s multifaceted process for monitoring 
program compliance, student outcomes and quality educational practices. The five phases of 
North Carolina’s revamped monitoring process (modeled on OSEP’s CIMP) were discussed: 

1 . self assessment; 

2. validation data collection; 

3. continuous improvement planning and implementation; 

4. verification; and 

5. continuous self-directed improvement and focused review. 

The system incorporates features of North Carolina’s former program compliance audit system 
allowing for verification of child counts and expenditures, teacher certification and other 
compliance requirements. Presenters shared the history of the new system’s development 
including stakeholder input, pilot testing in seven districts in 1999 and creation of a related data 
management system (currently in process). This system also incorporates a significant focus on 
student outcomes, including review of the data on participation and performance on assessments, 
suspension/expulsion, graduation and dropout rates and post-school outcomes. The session 
closed with discussion of issues and next steps (the presenter noted that there are school systems 
entering phase three) including, on-going motivation of stakeholders and evaluation and 
refinement of the system following full completion of the five-year cycle. 

Florida ’s Early Intervention Program: Continuous Improvement Monitoring Process 

Presenter: Renee Jenkins . Operations Review Specialist, Department of Children’s 
Health, FL (Part C) 

Last year, Florida’s Early Intervention program piloted a new Continuous Improvement 
Monitoring Process, designed to make monitoring more responsive to outcomes for children and 
families. Program requirements have been grouped into 1 5 components that are ranked according 
to those more directly related to positive outcomes. These components are the basis of 
monitoring. All 16 local programs develop a community plan incorporating the 15 components. 
Monitoring includes two levels — a desk review and a comprehensive on-site review. During 
monitoring, an EXCEL program is used that allows monitors to record whether or not a program 
is in compliance with the 15 components. A report is developed from this information that serves 
as the beginning of a corrective action plan. The state works with local programs to develop an 
approvable corrective action plan and a specific technical assistance plan. Every non-compliant 
issue does not receive a TA plan, only those that are persistent concerns. 

After monitoring, local and state data are analyzed to clarify identified issues, develop new data 
elements, update policies and revise training and technical assistance. For example, the state has 
developed a policy on the natural environments service delivery model, conducted statewide 
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dispute resolution training and developed strategies to reduce service coordinator caseloads. 
Incentives are included in the new monitoring process and are based on overall compliance, 
compliance by components and most improved performance. Consequences are natural 
consequences and include targeted site visits, written performance agreements, annual quality 
assurance reviews and cancellation of contracts. The monitoring process is ongoing and is 
evaluated by monitoring results, examining data trends and surveying monitoring participants. 

Focused Monitoring Part 2: Continuous Role of Stakeholders, Types and Selection of LEAs, and 
Remaining Data Issues 

Presenters: W. Alan Coulter. Director, National Center for Special Education 
Accountability Monitoring, LSU Health Sciences Center 
Mary Hudler, Assistant to the Director of Special Education, CA 
Eileen Amy , Administrator, Bureau of Instructional Support and Community 
Services, FL 

Virginia Beridon, Director of Special Education, LA 
Trudy Potts , Director of Focused Monitoring, NM 
Chris Drouin. Special Education Consultant, CA 
Lai it Roy , Special Education Section Chief, CA 

Karen Denbroeder. Program Supervisor, Bureau of Instructional Support and 
Community Services, FL 
Denise Stewart . Consultant, FL 

Batva Elbaum , Research Assistant Professor, University of Miami, FL 
Liz Leger, State Coordinator for Focus Monitoring, LA 
Donnalee Ammons . Monitoring Consultant, LA 

Alan Coulter began the second of three sessions on focused monitoring by providing a definition 
of focused monitoring and an overview of the three focused monitoring presentations. The 
purpose of the three sessions was to provide information about the activities of four states (CA, 
FL, LA, and NM) that are implementing focused monitoring. The visual presentation from these 
sessions is available online at the special education accountability website: www.hdc.lsuhsc.edu . 
This second session focused on the continuous role of stakeholders, type and selection of LEAs 
and remaining data issues. 

California uses key performance indicators and other data to customize and focus LEA 
monitoring. LEAs are required to design a method for obtaining parent input. The SEA specifies 
minimum questions to be addressed, a minimum level of participation and the need for special 
language groups, etc. LEAs may obtain parent input via focus groups, parent input meeting 
and/or surveys. 

New Mexico has a diverse stakeholder group. This group has met since 2001, developed key 
performance indicators and selected four “focus” indicators — least restrictive environment, 
evaluation/identification of specific learning disabilities, student performance and test 
participation. Based on the four focus indicators, LEAs fall into one of three groups for 
monitoring purposes — random, focused, or continuous improvement. The process for 
determining how LEAs are categorized was discussed. 




Monitoring: Proceedings of the 2002 National Monitoring Conference 
Project FORUM at NASDSE 



30 



Page 24 
February 2003 



Louisiana stakeholders play an important role in Louisiana’s focused monitoring process. They 
select the key indicators, decide on the tools to be used in the process and select sites to receive 
monitoring visits. Louisiana has learned that teamwork is the best approach. This year’s key 
indicators are high school diploma rate, placement settings and assessment participation and 
performance. The state is divided into four population groups; the LEA that scores lowest on the 
indicators in each group receives a monitoring visit. Additionally, 10 percent of the LEAs are 
chosen randomly for a visit. Exemplary systems are also visited (those that receive top scores in 
their population group). 

Florida obtain input from stakeholders (including parents, teachers and students) via workgroups, 
surveys/questionnaires and interviews. A monitoring workgroup with diverse representation 
helped develop the new monitoring system. They developed the following four key data 
indicators: percentage of students with disabilities in the regular education classroom, graduating 
with a regular diploma and participating in state assessment; and dropout rate for students with 
disabilities. Monitoring processes include focused, random, or self- assessment/continuous 
improvement. Continuous evaluation information will change over time. Key principles for 
stakeholder participation in the monitoring process include using multiple methods and data 
sources. 

Using Data Analysis to Enhance the Monitoring Process 

Presenters: Pamela Baker . Manager, Division of Accountability Development and 

Support, TX 

Lisa Gregg. Program Specialist, Division of Accountability Development and 

Support, TX 

Texas has over 1,200 school districts and charter schools. Responsibilities for special education 
are distributed throughout the SEA rather than located in one division. Monitoring across 
programs is conducted by the Department of Quality, Compliance and Accountability Reviews. 
The Texas Education Agency (TEA) has established good communication and collaboration 
strategies to effectively integrate requirements within the state's systems. TEA is in the process 
of expanding this collaboration across departments within TEA. For example, the state's agency- 
wide monitoring system is in the process of compiling data from a variety of TEA departments 
(e.g., financial audits and child nutrition) and soon all monitoring data will become part of the 
state's new automated reporting system. This automation will significantly increase the 
efficiency and accuracy in both gathering and reporting district data. 

The TEA analyzes a district’s status based on its results in the statewide assessment system, the 
Special Education Data Analysis System, special education complaints and due process 
hearings. The Special Education Data Analysis System currently considers 12 elements in 
district performance (e.g., federal and state risk factors, performance on the state assessment 
system and dropout rate). Complaints and due process hearings are tracked and analyzed by the 
state’s Division of Complaints and the Division of Legal Services and data are provided to the 
Department of Quality, Compliance, and Accountability Reviews to be included in the selection 
process for monitoring. District visits may be made for the following reasons: compliance on-site 
reviews, corrective action reviews, non-public facility reviews (when a district wants to place a 
student in such a facility) and other special investigations. 
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Common components for all on-site visits include establishing monitoring criteria, gathering 
data, data analysis, decision making and feedback. Compliance is judged on approximately 350 
items that target systemic issues, child-specific findings and corrective actions. Assessing quality 
and effectiveness occur through input from roundtables of district staff and parents, campus 
visits, case studies and staff interviews. 

The yearly demand for 200 on-site visits exceeds the capacity of the permanent staff (5 FTE), so 
additional support is outsourced (35 contracted monitors) and the Texas Schools Improvement 
Initiative provides peer reviewers. An innovative on-line training for these reviewers has proven 
successful and resource efficient. The TEA cannot provide technical assistance under state law; 
this function resides with the Regional Educational Service Centers, whose staff members 
participate in the monitoring training provided by the state. Other resources for districts in the 
monitoring process are maintained on the Department’s website. The Automated District 
Effectiveness and Compliance Reporting System will be a comprehensive tool in the near future 
to help the TEA use data and computers to make the monitoring system more accurate and 
efficient. 

Exploring the Relationship Between Monitoring Local Early Intervention and Preschool 
Programs and Providing Technical Assistance 

Presenters: Tal Black. Associate Director, NECTAC 
Renee Jenkins . Part C Program, FL 
Anne Lucas. Part C Coordinator, VA 
Maria Svnodi. Section 619 Coordinator, CT 
Carolyn Stiles. Part C Program, NH 

Representatives from four states described the focus of their monitoring and TA, with the 
ultimate goal of sustaining the quality of local programs. 

Florida : The Department of Health, the Part C lead agency, contracts with 16 local programs, 
which, in turn subcontract with direct service providers. The monitoring system includes an 
annual desk review of 15 components. Each program receives a comprehensive on-site review at 
least every three years. Florida’s technical assistance system is designed to promote effective 
systems change and improve the quality, compliance and results of services and supports. The 
TA liaison works with other lead agency staff to devise a compliance and quality improvement 
TA plan. The plan describes the recipients, methods of TA and timelines. The TA system 
contains tracking and evaluation components for continuous review and improvement. 

New Hampshire : The Part C program is located in the Department of Health and Human 
Services. The state agency contracts with 12 area agencies, which in turn subcontract with 23 
local programs. The state’s continuous quality improvement process begins with regional self 
assessment of data collected through local programs, followed by analysis of those data by 
regional teams and development of an improvement plan that is submitted to the state. TA is 
based on needs identified through the self assessment and compliance monitoring phases, 
provided to individual programs and statewide, from a variety of sources, such as professional 
groups, trainings, focused TA by Part C state staff, etc. 
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Virginia : The Part C system is decentralized, with nine state agencies and 40 local Interagency 
Coordinating Councils (ICCs). Common elements in the state’s monitoring and TA systems are: 
focus on results; data-driven decision making; locally-driven process; and building local 
capacity. The monitoring system contains self assessments, state verification reviews and logic 
model analyses to develop Plans of Improvement. TA is provided during all stages and 
consultants from both the monitoring and TA sections coordinate to ensure integrated and 
systems- focused local plans. Maintaining relationships between the TA and monitoring 
components continues to present challenges, but the systems are positively regarded by local 
entities and state staff. Focused monitoring considerations are being discussed for 2004. 

Connecticut : The state has six regions and 158 LEAs. Each region is monitored at least once 
every six years. All LEAs in the region are monitored and receive either a site visit or a portfolio 
review (i.e., desk audit, but no site visit). Decisions regarding site visits are based on a set of 
triggers. Occasionally a LEA is selected that is not in the region being monitored due to concern, 
for example, about data or complaints. Monitoring results and data are linked to CSPD, SIG, 
general supervision, TA and training initiatives. The state is introducing focused monitoring. The 
predominance of focused monitoring on issues for older students (e.g., transition, dropout and 
graduation) may contribute to less visibility for the preschool population. "If we don't ask for 3-5 
data, the population potentially loses importance;" hence the description of the state’s preschool 
program as the "golden gate between two pieces of land." The Part B 619 program provides 
facilitation/connection between Part C and entering Part B school-age programs. 

Tal Black asked the panel of states to describe the benefits and challenges of maintaining close 
relationships between monitoring and TA. Benefits included improved effects of targeting 
assistance to monitoring issues, improved systemic outcomes and improved accuracy and 
consistency. Challenges ranged from dueling priorities, alternating “hats,” inadequate time and 
variety in expertise. Further information and details on state early childhood TA systems can be 
found on the NECTAC website, http://www.nectac.oro/topics/statetasvs/statetasvs.asp . 

Tools for Accountability and Monitoring 

Presenter: Jean Taylor, Quality Assurance Coordinator, ID 

Idaho has focused on increasing monitoring efficiency and motivating districts to produce 
systemic change by matching similar districts, setting equitable goals and rewarding improved 
outcomes for students. In the past three years the state has moved toward an outcome-based 
process and more focused monitoring with a strong emphasis on the use of data. This has 
allowed for better allocation of resources in a large state with a small DOE staff. It also has 
provided an incentive for districts to begin using the data they report to the SEA. 

Tools for Accountability and Monitoring: Districts report data annually to the state special 
education director, district superintendent and the public via the state’s website. Goals on 
indicators are established by matching similar districts. A monitoring decision matrix is used that 
outlines areas of possible concern and areas where more information is needed. District self- 
evaluation is conducted one year before state monitoring. Districts develop a plan for improving 
results. A tiered monitoring approach is used with three options: full review, focused review or 
mini review. 
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Advantages of this new process include partnership between the state and districts in meeting 
goals, fair expectations, improved outcomes for kids, allocation of resources to areas of greatest 
need in the state and change of focus from process to outcomes. 

Kansas Continuous Improvement Monitoring: Documenting Data-Driven Decision Making in 
Kansas Schools 

Presenters: Carol Dermyer , Coordinator, Student Support Services Team, KS 
Nancy Gray . Education Program Consultant, KS 
Deb Chappell. Technical Assistance Project Coordinator, KS 
Crystal Davis . Program Evaluations Specialist, KS 
Kevin Davis. Program Evaluations Specialist, KS 

During the Kansas Continuous Improvement Monitoring (CIM) cycle, local agencies engage in 
data collection, data analysis, improvement planning, implementation of improvement strategies 
and progress monitoring. Guidance for these activities is provided in the Kansas CIM Workbook 
and self-assessment document. It is the expectation that CIM will enhance program improvement 
and result in better services and supports for students with exceptionalities, allowing them full 
access, participation and progress in the general curriculum. 

The CIM Self Assessment is the summary of each LEA’s compliance with special education 
laws and regulations. It documents the effectiveness of each LEA’s special education and related 
services based on student performance and outcomes. The self assessment contains 66 indicators 
that each LEA must address during the first year of its CIM cycle. These indicators can be sorted 
by both the type of data they require and the topics they address. For each indicator, the LEA 
gathers the appropriate data, analyzes it and then determines its level of performance using a 
performance level rubric. Indicators are ranked as strength, meets requirements, needs 
improvement or non-compliant. The self-assessment process is driven by data — performance 
data, survey data, observation data and data from review of student files. LEAs identify 
improvement strategies for indicators needing improvement or those that are non-compliant. 
Data are reported annually to document whether they are showing improvement. 

LEAs receive support and technical assistance throughout the CIM process from a CIM 
facilitator, technical support staff, process overview training, an on-site visit team member 
training and legal requirements database training. Upon completion of the self assessment, each 
LEA is encouraged to align its Continuous Improvement Application (CLA) with any CIM 
indicators identified as needing improvement or being non-compliant. 

Focused Monitoring Part 3: On-Site Processes, Report Writing, and Corrective Actions 
Presenters: W. Alan Coulter. Director, National Center for Special Education 
Accountability Monitoring, LSU Health Sciences Center 
Mary Hudler. Assistant to the Director of Special Education, C A 
Eileen Amy. Administrator, Bureau of Instructional Support and Community 
Services, FL 

Virginia Beridon , Director of Special Education, LA 
Trudy Potts . Director of Focused Monitoring, NM 
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Chris Drouin, Special Education Consultant, CA 
Lalit Roy . Special Education Section Chief, CA 

Karen Denbroeder, Program Supervisor, Bureau of Instructional Support and 
Community Services, FL 
Denise Stewart , Consultant, FL 

Batva Elbaum. Research Assistant Professor, University of Miami, FL 
Liz Leger. State Coordinator for Focus Monitoring, LA 
Donnalee Ammons , Monitoring Consultant, LA 

Alan Coulter began with a definition of focused monitoring and an overview of the three focused 
monitoring sessions. This third session provided information about the activities of four states 
(CA, FL, LA and NM) implementing focused monitoring. The visual presentation from these 
sessions is available online at the special education accountability website: www.hdc.lsuhsc.edu . 
(Click on National Monitoring Conference in Salt Lake City.) This third session focused on the 
on-site process, report writing and corrective actions. 

Florida : Prior to going onsite, information is gathered through parent, teacher and student 
surveys; review of student records and district forms; and a pre-staffing for the SEA staff. On 
site, the team conducts individual interviews; holds parent, teacher and student focus groups; and 
conducts student case studies and classroom observations. Activities take place at the district and 
school level. Reports always address staff knowledge and training, stakeholder opinion and the 
factors associated with the four key indicator “triggers.” A system improvement plan is 
developed based on the key issues determined by the monitoring team. 

Louisiana : The pre-staffing includes all members of the monitoring team, which does not include 
any SEA staff. Peers from other districts, parents and university personnel conduct the on-site 
visit. During the visit, the team can change the direction of the investigation based on what is 
learned and discussed at daily debriefings. At the end of the visit, the team must arrive at 
consensus if there is a finding of systemic non-compliance. The team leader conducts a brief exit 
meeting. Reports are delivered to the LEA within 60 days. The corrective action plan includes 
timelines and personnel responsible for items on the plan. The LEA must provide "evidence of 
change.” Contact with the SEA during this time is ongoing. A follow-up visit by the team leader 
is made one year later. 

California : In preparing for a focused monitoring visit in California, data from the key 
performance indicators for the LEA are compiled and reviewed along with the results of parent 
focus groups and surveys. Elements reviewed include on-site, self-reviews, findings, corrective 
actions and tracking information. Standard and customized on-site and self-reviews are 
conducted. Compliance includes both procedural guarantees and educational benefits. A new 
review component is being used this year to determine if services are reasonably planned to 
result in education benefit for students. 

New Mexico : The selection of LEAs to monitor in New Mexico is based on performance on the 
indicators and the type of LEA (focused, random, or continuous improvement). The on-site team 
does not include SEA staff. The team leader is a retired educator, with three to five members 
who are peers from other districts and one to two members who are parents. The visits range 



O 



Monitoring : Proceedings of the 2002 National Monitoring Conference 
oject FORUM at NASDSE 



Page 29 
February 2003 



from three to five days, including a parent information meeting on day one. The rest of the visit 
includes interviews, file reviews and classroom observations. The report is drafted and revised 
by the team leader and edited and finalized at the SEA. A systemic finding of non-compliance 
must be corroborated by two to three sources. The SEA determines "evidence of change" 
measures. The LEA then drafts its improvement plan with guidance from the SEA. The SEA 
must also approve the plan. Ongoing assistance is provided with a follow up visit by the SEA 
consultant one year later. 

Post-Conference Plenary: 

OSEP Monitoring: Continuous, Results-Oriented and Focused 

Presenters: Ruth Ryder . Division Director, Monitoring and State Improvement 
Planning, OSEP 

Larry Wexler. Deputy Director, Monitoring and State Improvement 
Planning, OSEP 

Sarah Willis. Part C, Monitoring and State Improvement Planning, OSEP 

The visual presentation from this session is available online at the National Monitoring and Best 
Practices Website (look for the National Monitoring Conference link) at: 

http://www.dssc.org/frc/nmpp/index.htm . 

This session provided information on the direction OSEP monitoring is moving — the next phases 
of CIMP. This is the transition year and MSIP will review what the monitoring process means 
for OSEP and the states. 

OSEP’s monitoring process has evolved over time. Many changes have been made based on 
input from the national monitoring stakeholder group, including: 

• public awareness and involvement — monitoring has become a more open process, with 
steering committees playing a central role; 

• focus — monitoring is moving from compliance to a results focus; 

• scope of OSEP data collection — data collection has become more focused on critical areas; 

• accountability — has shifted to the states; 

• improvement — the monitoring focus has moved from process to improvement for students; 

• technical assistance and support — the RRCs and NECTAC have moved toward a more 
holistic approach to providing TA and support; and 

• consequences — have shifted from corrective action plans (compliance-oriented) to 
improvement plans (results-focused). 

Future changes in OSEP monitoring will include involving communities and the public more in 
the process; focusing on critical outcomes and indicators; using a structured process to select 
states for OSEP intervention; providing purposeful integration of technical assistance to states; 
and using predictable rewards and sanctions. The current challenge is how to take the best of 
CIMP and best of focused monitoring and develop an evidence-based, effective, timely and 
transparent federal monitoring process. 
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The best of CIMP includes: 

• steering committee involvement; 

• use of self assessment (evidence-based) to determine areas needing improvement; 

• use of improvement plans to guide change activities; 

• states taking ownership and accountability for improvement; and 

• focusing on results for children and families as well as compliance. 

The best of focused monitoring includes: 

• critical outcome and indicator data used to focus improvement activities; 

• data and information made public (similar to CIMP); 

• evidence-based information used to allocate OSEP resources; 

• clear, predictable triggers used for determining interventions and sanctions; and 

• OSEP providing support to states in need of improvement. 

In order to meet this challenge, OSEP has conducted a cross walk of the CIMP cluster 
components and indicators, focused monitoring priority areas and indicators and GPRA 
objectives and indicators, resulting in a simplified set of child and family outcomes and 
indicators. Additionally, characteristics of effective state systems have been developed (these 
will be rolling out soon) and OSEP leverage points for change have been identified. Bringing 
together CIMP and focused monitoring, a model has emerged that has three main components: 
(a) identification — state selection for intervention; (b) analysis — a closer look behind the 
data/drill down; and (c) improvement — planning and implementing to leverage change. 

Identification : OSEP will be using rank order data. This is primarily the 618 data (used for the 
Annual Report to Congress) and some Biennial Performance Report (BPR) data. OSEP also 
wants to eventually use data that are not yet available (e.g., data on family capacity, family 
involvement and post school outcomes). Specifically for Part B, OSEP will use 618 data on 
exiting and educational environments. Specifically for Part C, OSEP will use data on 
identification and settings. 

Analysis'. Once a state is identified for intervention, OSEP will work with the state to go deeper 
into the data, breaking information down by race/ethnicity, disability, region, etc. In addition, 
data will be examined from the self assessment, improvement plan and public input. 

Improvement : OSEP will work with states to develop an aggressive evidence-based 

improvement plan. The plan must include timelines and benchmarks to measure improvement. 
OSEP will review and approve the plan. If there is improvement, then the state will continue to 
work on improvement activities. If the state does not meet the benchmarks or timelines, then 
OSEP will further analyze the problem (e.g., Were strategies implemented? Are there political 
reasons progress was not made? Were strategies ineffective? ). 

Using 1999-2000 data and the priorities, indicators, benchmarks and triggers developed by the 
national monitoring stakeholders group, Sarah Willis gave a practical example of how this 
process would work for Part C. Larry Wexler gave a practical example of how this process 
would work for Part B. Priorities are key elements of IDEA. Indicators measure whether a 
priority has been met. Benchmarks are the level of expected state performance relative to the 
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indicators. Triggers are used to determine when OSEP would intervene. Based on trigger levels 
states are placed in one of four categories: 

Category 1 — meeting or exceeding the benchmarks 

Category 2 — below benchmarks, but should be able to meet benchmarks relatively quickly 
Category 3 — significantly below benchmarks, needs intervention to meet benchmarks 
Category 4 — critically below benchmarks, needs intensive intervention to meet benchmarks 

Part C example : 

Priority — all infants and toddlers are appropriately identified 
Indicator — the percentage of infants and toddlers with IFSPs 
Benchmark — at least 2% of infants and toddlers have IFSPs 
Category 1 — at least 2% of infants and toddlers have IFSPs (22 states) 

Category 2 — less than 2% but more than 1.75% of infants and toddlers have IFSPs (9 states and 
PR) 

Category 3 — less than 1.75% but more than 1.5% of all infants and toddlers have IFSPs (5 
states) 

Category 4 — less than 1 .5% of infants and toddlers have IFSPs (14 states and DC) 

OSEP will intervene with states falling into Category 4. OSEP will work with these states to go 
deeper into the data, conducting a “drill down.” For this indicator, OSEP would look at the 
state’s eligibility definition, trend data, whether children with IFSPs represent the state 
population (race/ethnicity, income, rural/urban, migrant, etc.), how well child find is coordinated 
with other state agencies, referral sources and public awareness materials. 

Part B example : 

Priority — inclusion of children and youth with disabilities in typical community and school 
settings with their nondisabled peers with needed supports 
Indicator — percentage of children with disabilities outside regular education class less than 

20% of the school day; percentage of children outside regular education class greater than 
60% of the school day; percentage of children in separate school/facility. 

Benchmark — 90% of children with disabilities will be educated in general education classes for 
80% or more of the school day 

OSEP will intervene with states falling into Category 4. In the drill down, OSEP will look at 
more recent data, trend data, factors that might skew data (e.g. large population centers), 
race/ethnicity, state policies and procedures around an issue. Developing a data infrastructure is 
critical to this process. The purpose of the “drill down” is to focus state monitoring efforts, 
identify problems and develop improvement strategies. 

Ruth Ryder presented several remaining challenges to this process, including: 

• lack of data for some priorities; 

• accuracy of data — there are no real time data, so the best data available will be used (Westat 
will work with states to ensure the data are as accurate as can be); 

• limited technical assistance resources — funds are not available to work with every state; 
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OSEP would like to put more money in Part D; 

• implementation of a results-oriented system within a process-oriented law (e.g., there is 
nothing in the law that says states must serve two percent of infants and toddlers or that 
graduation rates should be at a certain level). 

Participants broke into small groups by RRC regions to generate questions regarding this new 
monitoring approach. Each group was asked to bring their top questions to the larger group for 
discussion. Below is a summary of the large group question and answer session. 

Q: Why doesn't OSEP collect gender data? 

A: OSEP does not have the authority to do so. At the time of reauthorization, OSEP will provide 
input on additional data it would like to collect. 

Q: LRE is a process not an outcome; can this be taken into account? 

A: LRE data are viewed as a proxy for participation in the general education curriculum. It is 
also part of a civil rights requirement. As better performance data are collected, OSEP hopes to 
cross walk LRE data in states with performance data. 

Q: There are a number of questions about graduation and dropout data — each state has different 
definitions and requirements, how can OSEP compare data across states? 

A: In order to try and address this concern, general education graduation rates were used and 
states were ranked based on their difference between general and special education graduation 
rates. Another problem is that this is a moving target because graduation requirements change. 
Remember these rankings do not put a state out of compliance; rather it is a catalyst for OSEP to 
take a closer look at what is going on. 

Q: Is there any movement toward common definitions of benchmarks, triggers, etc.? 

A: OSEP hopes that definitions will be developed as part of the work of the new national 
monitoring center. 

Q: What is the research base for setting the Part C child find benchmark at two percent of 
infants and toddlers with IFSPs? 

A: The two percent was established as a GPRA indicator and was based on CDC (Centers for 
Disease Control) data. The CDC tracks diagnoses and has found that 1.5 percent of this 
population, at a minimum, is diagnosed with a specific condition or syndrome. Depending on a 
state's definitions, at least another .5 percent would be identified. This information is not 
available in writing. 

Q: If there is a focus on the number of children served in Part C, where should the funding come 
from? 

A: For this particular indicator, finance is not considered as part of the “drill down” because 
when a state draws down funds it ensures that child find will happen. Funding may be a factor 
for other issues (e.g. state funding formula encouraging more restrictive placements). 

Q: Are there specific indicators for Section 619? 

A: Section 619 issues fall into both B and C indicators. During “drill downs,” OSEP will look at 
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issues in common. For example, if Part C transition data does not look good, OSEP will look at 
both Part C and 619 information. Many of the Part B indicators apply to 619 and they will be 
looked at in that way. 

Q: To what extent has OSEP ensured research efforts are coordinated with expected outcomes? 
A: A next step regarding child and family outcome data is have one of the OSEP research 
projects take a look at these issues. 

Q: What if OSEP picks different priority areas than the state chooses? 

A: This would be a concern if the state has low performance in an area and has not chosen it as 
part of their improvement plan. OSEP would then want to take a look at why the concern is not 
being addressed. 

Q: How does OSEP take into account where states are in their improvement planning? 

A: OSEP views the process as continuous — no state will ever be done with improvement 

planning. If a state is low performing in an area, OSEP will do a drill down to see what’s being 
done in improvement planning and address concerns. 

Q: Is there a maximum number of areas that a state could be targeted for intervention? 

A: The number of areas is limited to two for Part B. 

Q: Is OSEP looking at trend data? 

A: OSEP asked Westat to run trend data and these data will be considered. 

Q: Considering the differences across states, how can states be ranked? 

A: The whole point of focused monitoring is to focus resources. OSEP must identify states that 
need the most help. In order to this, comparisons are made across states, but trend data are used 
in the drill down. 

Q: What is the intent of using BPR data? 

A: NCEO is doing an analysis of some BPR data — participation and performance on assessment. 
The Center’s staff has developed a way to rank this information, but OSEP recognizes that states 
are new at reporting these data. There is a department-wide group working to develop a single 
data collection for all programs in the department. This is in the very early stages. OSEP will 
continue to collect Section 618 data and would like to add performance data to the requirement. 
Then the BPR would be more an analysis than a report. 

Q: What sanctions and rewards will OSEP use? 

A: Sanctions will include designating a state as high risk; conditional grant approval; compliance 
agreements; cease and desist orders; withholding funds in part or in whole. Rewards include 
recognition as the primary reward (e.g., a letter to the governor and notice on the OSEP website). 
OSEP does not have the authority to attach money, but perhaps might consider awarding a small 
grant so the state could mentor other states or produce documents. 

Q: How does focused monitoring affect what SEAs hold LEAs accountable for? Will states be 
cited for not monitoring districts if only focused monitoring is used? 
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A: OSEP does not have authority to tell states how to monitor (e.g., if a state visits/monitors 
every program every five years). If OSEP found a district that had a lot of problems and the state 
had not identified it, OSEP would look at the system to find out why that happened. 

According to Ruth Ryder, 2001 data will be available soon and a rank ordering will be done. 
This information should be sent to the states in January, 2003. 

Transition to this new process will include: 

• aligning Part C Annual Performance Reports and Part B BPRs with CLMP self assessment 
and improvement plan reporting; 

• validating child and family outcomes and indicators; 

• identifying high-risk indicators; 

• clarifying vocabulary (triggers, outcomes, etc); 

• sending data out to states for review; 

• developing technical assistance resources with OSEP’s Research to Practice Division; 

• defining roles for OSEP, RRCs and NECTAC, the Monitoring Center, and other OSEP- 
funded technical assistance centers; 

• revising and issuing a new monitoring manual (hopefully by next summer); 

• conducting summer institutes in 2003; and 

• developing a communication plan with states and the public. 

Additional items to note: 

• 2002 grant award letters will go up on OSEP’s website soon. 

• There are plans to do collaborative monitoring in conjunction with NCLB and Vocational 
Rehabilitation Act. 

• A third party evaluation of state monitoring is under consideration, including how it has been 
impacted by OSEP’s monitoring (i.e., how has OSEP’s general supervision of states 
impacted states’ general supervision of LEAs). 

In closing, remember change is like a curve in road, it is not a problem unless you forget to turn. 
We are all going to learn how to do this together. 



Presentations at the Sixth National Monitoring Conference illustrate the ever-evolving 
monitoring practices at both the state and federal level. Presentations by OSEP staff provided an 
overview of the upcoming changes in OSEP monitoring, namely the integration of CIMP and 
focused monitoring. With regards to the development of a federal focused monitoring model, the 
work of the national stakeholders group was shared in several presentations. 

State-level presentations portrayed state monitoring systems that have successfully incorporated 
elements of OSEP’s CIMP, including the use of steering committees and stakeholders, data- 
based decision making, the use of self-assessment tools, integrating technical assistance and 
monitoring, and accountability (development of indicators and outcome measures). With the 
strong emphasis on accountability, data issues were the primary focus in several presentations 
and were discussed in several others. Data topics discussed included data collection, 
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development of data systems, data management, use of data, and concerns with the accuracy and 
reliability of data. 

Since the last National Monitoring Conference in 1999, several resources have emerged that 
provide information specifically about monitoring in special education. These include: 

National Monitoring and Promising Practices Website 

http://www.dssc.org/frc/nmpp/index.htm . 

OSEP Monitoring Website (U.S. Department of Education) 

http://www.ed.gov/ofrices/OSERS/OSEP/Monitoring/ 

National Center for Special Education Accountability Monitoring website 
www.hdc.lsuhsc.edu (click on Special Education Accountability Consortium) 
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